There are many situations in life where we simply don't know which tool to use for a particular job. For example, consider a situation where we need to cut a piece of wood. We may not know everything about the piece of wood that we really need to know in order to select the correct tool to cut that piece of wood. We may also not know if we need to cut the wood along its grain or if we need to cut the wood across the grain. Those that are familiar with woodworking and those that are in the construction industry will immediately know that cutting along the grain requires a ripping blade whereas cutting across the grain requires a crosscut blade.
Once we know the placement of a cut, the grain of the wood may be obscured by a heavy coat of paint. Because the grain of the wood is obscured, we may not realize that we need to cut along the grain until we actually start cutting the wood. It is likely that we initially selected a crosscut blade because that is the most common cutting direction, especially in the construction industry. In this case, we would need to change to a ripping blade so that we can achieve a cleaner cut along the remainder of the cutting job. We also could be faced with an even bigger problem in a situation where the heavy coat of paint actually prevented us from realizing that the material under the paint is actually aluminum. This, of course, requires us to select a metal cutting blade.
This simple wood cutting analogy can be applied in many constructs. For example, in computing systems there are situations where we simply do not know all of the characteristics of an incoming set of information. Typically, we refer to such an incoming set of information as a “work unit”. A work unit, in various situations, consists of information that needs to be processed in order to obtain some useful result. Just as with selecting a particular blade for cutting wood, a computer system needs to select a particular solution strategy in order to effectively and correctly process an incoming work unit. This yields a solution unit that is based on the incoming work unit as processed by the properly selected solution strategy.
One example that is easy to understand is that of a computer system that processes data packets. For example, data packets could be in the form of Internet protocol packets. Or, the data packets could represent streaming data such as that used in Internet video broadcasting. There are a myriad of other types of data packets that could conceivably arrive as work units, which the computer system must process. Typically, the computer system will need to use a different solution strategy for various types of data packets in order to obtain a useful result.
Of course, if the computer system typically expects to see Internet protocol packets, the computer system applies a solution strategy by executing a particular computer program. This causes the computer system to examine an Internet protocol packet, process the packet in order to understand any packet specific delivery requirements and then direct the packet to an output. However, if the arriving data packet is not an Internet protocol packet, the computer system will still process the incoming packet as an Internet protocol packet and this would yield an undesirable result.
Once the computer system realizes that it selected the wrong solution strategy, it will attempt to process the incoming data packet according to a different solution strategy. The computer system will try different solution strategies until it finds the right strategy and is able to correctly process an incoming data packet. All of these attempts to properly process an incoming data packet are, up until now, performed in a serial manner. This really limits the number of incoming data packets that can be processed over a particular amount of time.
Modernly, a computer system uses parallel processing cores in order to reduce the amount of time that is spent trying different solution strategies. This type of modern parallel computing system includes a plurality of processing cores, where various processing cores are configured to apply different solution strategies to an incoming work unit. Then, the first processing core that develops an acceptable result is allowed to send its result to a subsequent process. As in the earlier presented example of processing incoming data packets, the first processing core that develops an acceptable result is allowed to send its result to an output, for example by sending a processed data packet to the Internet.
This technique of using parallel processing cores that are configured to apply a different solution strategy to an incoming work unit is often referred to as thread level speculative (TLS) processing. Up until now, thread level speculative processing was managed by a collection of software elements executed by the various processing cores included in a parallel processing system.